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Highlights 


Why  GAO  Did  This  Study 

Department  of  Defense  (DOD) 
weapon  programs  often  experience 
significant  cost  and  schedule 
problems  because  they  are  allowed 
to  start  with  too  many  technical 
unknowns  and  not  enough 
knowledge  about  the  development 
and  production  risks  they  entail. 
GAO  was  asked  to  review  the 
department’s  Analysis  of 
Alternatives  (AO A)  process — a  key 
first  step  in  the  acquisition  process 
intended  to  assess  the  operational 
effectiveness,  costs,  and  risks  of 
alternative  weapon  system 
solutions  for  addressing  a  validated 
warfighting  need.  This  report 

(1)  examines  whether  AOAs  have 
been  effective  in  identifying  the 
most  promising  options  and 
providing  a  sound  rationale  for 
weapon  program  initiation, 

(2)  determines  what  factors  have 
affected  the  scope  and  quality  of 
AOAs,  and  (3)  assesses  whether 
recent  DOD  policy  changes  will 
enhance  the  effectiveness  of  AOAs. 
To  meet  these  objectives,  GAO 
efforts  included  collecting 
information  on  AOAs  from  32 
major  defense  acquisition 
programs,  reviewing  guidance  and 
other  documents,  and  interviewing 
subject  matter  experts. 


What  GAO  Recommends 


GAO  is  recommending  that  DOD 
establish  criteria  and  guidance  for 
how  AOAs  should  be  conducted 
and  ensure  AOAs  are  completed 
before  requirements  are  set.  DOD 
concurred,  and  stated  that  recently 
revised  policies  improve  AOAs.  We 
believe  that  the  policy  revisions  do 
not  go  far  enough  to  address  our 
recommendations. 

View  GAO-09-665  or  key  components. 

For  more  information,  contact  Michael  J. 
Sullivan  at  (202)  512-4841  or 
sullivanm@gao.gov. 


What  GAO  Found 

Although  an  AOA  is  just  one  of  several  inputs  required  to  initiate  a  weapon 
system  program,  a  robust  AOA  can  be  a  key  element  to  ensure  that  new 
programs  have  a  sound,  executable  business  case.  Many  of  the  AOAs  that 
GAO  reviewed  did  not  effectively  consider  a  broad  range  of  alternatives  for 
addressing  a  warfighting  need  or  assess  technical  and  other  risks  associated 
with  each  alternative.  For  example,  the  AOA  for  the  Future  Combat  System 
program,  one  of  DOD’s  large  and  most  complex  development  efforts,  analyzed 
the  operational  performance  and  cost  of  its  alternatives  but  failed  to  compare 
the  technical  feasibility  and  risks,  assuming  that  the  technologies  would 
perform  as  forecasted.  Without  a  sufficient  comparison  of  alternatives  and 
focus  on  technical  and  other  risks,  AOAs  may  identify  solutions  that  are  not 
feasible  and  decision  makers  may  approve  programs  based  on  limited 
knowledge.  While  many  factors  can  affect  cost  and  schedule  outcomes,  we 
found  that  programs  that  had  a  limited  assessment  of  alternatives  tended  to 
have  poorer  outcomes  than  those  that  had  more  robust  AOAs. 

The  narrow  scope  and  limited  risk  analyses  in  AOAs  can  be  attributed  in  part 
to  program  sponsors  choosing  a  solution  too  early  in  the  process,  the 
compressed  timeframes  that  AOAs  are  conducted  under,  and  the  lack  of 
guidance  for  conducting  AOAs.  While  AOAs  are  supposed  to  provide  a  reliable 
and  objective  assessment  of  viable  weapon  solutions,  we  found  that  service 
sponsors  sometimes  identify  a  preferred  solution  or  a  narrow  range  of 
solutions  early  on,  before  an  AOA  is  conducted.  The  timing  of  AOAs  has  also 
been  problematic.  Some  AOAs  are  conducted  under  compressed  timeframes 
in  order  to  meet  a  planned  milestone  or  weapon  system  fielding  date  and  are 
conducted  concurrently  with  other  key  activities  required  to  become  a 
program  of  record.  This  can  short-change  a  comprehensive  assessment  of 
risks  and  preclude  effective  cost,  schedule,  and  performance  trade  offs  from 
taking  place  prior  to  beginning  development.  Furthermore,  while  DOD  has  an 
opportunity  to  influence  the  scope  and  quality  of  AOAs,  it  has  not  always 
provided  guidance  for  conducting  individual  AOAs. 

Recognizing  the  need  for  more  discipline  in  weapon  systems  acquisition,  DOD 
recently  revised  its  overall  acquisition  and  requirements  policies.  If 
implemented  properly,  the  revised  policies  could  provide  a  better  foundation 
for  planning  and  starting  new  programs  with  sound,  knowledge-based 
business  cases.  Included  in  the  revised  acquisition  policy  are  several 
mechanisms  to  improve  the  AOA  process.  For  example,  the  policy  revisions 
should  help  ensure  that  DOD  direction  is  provided  before  AOAs  are  started 
and  that  they  are  conducted  at  an  early  point  in  the  acquisition  process  where 
their  results  can  inform  decisions  affecting  program  initiation.  While  these 
policy  changes  are  promising,  DOD  must  ensure  that  they  are  consistently 
implemented  and  reflected  in  decisions  on  individual  programs.  Furthermore, 
more  specific  criteria  and  guidance  for  how  AOAs  should  be  conducted  may 
need  to  be  developed  to  ensure  they  meet  their  intended  objectives  and 
provide  an  in-depth  assessment  of  alternatives. 
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United  States  Government  Accountability  Office 
Washington,  DC  20548 


September  24,  2009 

The  Honorable  John  F.  Tierney 
Chairman 

Subcommittee  on  National  Security  and  Foreign  Affairs 
Committee  on  Oversight  and  Government  Reform 
House  of  Representatives 

Dear  Mr.  Chairman: 

Cost,  schedule,  and  performance  problems  in  the  Department  of  Defense’s 
(DOD)  weapon  system  programs  are  serious.  Recently,  we  reported  that 
the  department’s  2008  portfolio  of  96  major  defense  acquisition  programs 
experienced  cost  growth  of  $296  billion,  experienced  an  average  delay  in 
delivering  initial  capabilities  of  22  months,  and  have  delivered  fewer 
quantities  and  capabilities  to  the  warfighter  than  originally  planned.1  Over 
the  past  several  years,  our  work  has  highlighted  a  number  of  underlying 
causes  for  why  poor  outcomes  have  occurred  in  weapon  programs.  One 
key  cause  is  that  DOD  allows  programs  to  begin  without  a  sound  match 
between  requirements  and  the  resources  needed  to  achieve  them.  That  is, 
programs  enter  the  acquisition  process  with  requirements  that  are  not  fully 
understood,  cost  and  schedule  estimates  that  are  based  on  optimistic 
assumptions,  and  a  lack  of  sufficient  knowledge  about  technology,  design, 
and  manufacturing. 

With  the  growing  fiscal  pressures  now  facing  the  nation,  DOD  needs  to  get 
the  best  value  for  every  dollar  it  invests  in  weapon  system  programs.  The 
department’s  management  of  its  weapon  system  programs  has  been  a 
matter  of  congressional  concern  for  many  years.  In  2008,  the 
Subcommittee  requested  that  we  study  how  DOD  makes  trade  offs  in 
requirements,  costs,  and  technical  risks  before  approving  programs  to 
start  development.  Specifically,  the  Subcommittee  asked  us  to  review  the 
department’s  Analysis  of  Alternatives  (AO A)  process — a  key  first  step  in 
the  acquisition  process  intended  to  assess  the  operational  effectiveness, 
costs,  and  risks  of  alternative  weapon  system  solutions  for  addressing  a 
validated  warfighting  need.  Determining  what  type  of  weapon  system  to 
pursue  is  critical  because,  according  to  a  recent  estimate,  about  three- 


1  GAO,  Defense  Acquisitions:  Assessments  of  Selected  Weapon  Programs,  GAO-09-326SP 
(Washington,  D.C.:  Mar.  30,  2009). 


Page  1 


GAO-09-665  Analysis  of  Alternatives 


quarters  of  a  program’s  total  life-cycle  cost  is  influenced  by  decisions 
made  before  it  is  approved  to  start  development.2  This  report  assesses 

(1)  whether  AO  As  have  been  effective  in  identifying  the  most  promising 
options  and  providing  a  sound  rationale  for  weapon  program  initiation, 

(2)  the  factors  that  affect  the  scope  and  quality  of  AO  As,  and  (3)  whether 
recent  DOD  policy  changes  will  enhance  the  effectiveness  of  AO  As. 

To  assess  DOD’s  AOA  process,  we  reviewed  relevant  DOD  and  military 
service  policy  and  guidance,  and  interviewed  officials  from  the  Joint  Staff, 
Office  of  Program  Analysis  &  Evaluation,  Air  Force  Office  of  Aerospace 
Studies,  Army  Training  and  Doctrine  Command  Analysis  Center,  and  other 
subject  matter  experts.  We  also  collected  and  analyzed  information  on 
AO  As  from  32  major  defense  acquisition  programs3  that  had  started  since 
fiscal  year  2003.4  We  obtained  information  from  program  officials  on  how 
these  AO  As  were  conducted  and  whether  the  AO  As  contributed  to 
changes  in  the  program’s  weapon  system  concept.  Furthermore,  we 
reviewed  AOA  documents  and  DOD  and  service  guidance  for  conducting 
these  AO  As.  In  reviewing  AO  As,  we  examined  the  scope  of  alternatives 
that  were  considered  and  the  extent  of  risk  assessments  conducted  for 
each  alternative.  This  work  was  conducted  from  June  2008  to  September 
2009  in  accordance  with  generally  accepted  government  auditing 
standards.  Those  standards  require  that  we  plan  and  perform  the  audit  to 
obtain  sufficient,  appropriate  evidence  to  provide  a  reasonable  basis  for 
our  findings  and  conclusions  based  on  our  audit  objectives.  We  believe 
that  the  evidence  obtained  provides  a  reasonable  basis  for  our  findings 


2  National  Research  Council,  Pre-Milestone  A  and  Early  Phase  Systems  Engineering:  A 
Retrospective  Review  and  Benefits  for  Future  Air  Force  Systems  Acquisition 
(Washington,  D.C.:  2008). 

3  Major  defense  acquisition  programs,  referred  to  as  acquisition  category  (ACAT)  I 
programs,  are  defined  as  those  that  are  expected  to  exceed  $365  million  (in  fiscal  year  2000 
constant  dollars)  in  research,  development,  test,  and  evaluation  (RDT&E)  or  $2.19  billion 
(in  fiscal  year  2000  constant  dollars)  in  procurement  costs.  For  ACAT  ID  programs,  the 
Under  Secretary  of  Defense  for  Acquisition,  Technology,  and  Logistics  serves  as  the 
Milestone  Decision  Authority  (MDA)  and  determines  whether  the  program  can  proceed  to 
the  next  phase  of  acquisition  while,  for  ACAT  IC  programs,  the  MDA  is  the  head  of  the 
DOD  component  or,  if  delegated,  the  service  Component  Acquisition  Executive. 

4  We  selected  programs  that  started  after  2003,  to  correspond  with  significant  revisions  that 
had  been  made  to  DOD’s  acquisition  policy  at  that  time. 
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Background 


and  conclusions  based  on  our  audit  objectives.  Additional  information 
about  our  scope  and  methodology  is  provided  in  appendix  I;  appendix  III 
contains  information  about  the  programs  and  AO  As  we  reviewed. 


Before  service  or  joint  sponsors  can  initiate  major  defense  acquisition 
programs  and  begin  system  development  at  Milestone  (MS)  B,  they  are 
required  by  DOD’s  acquisition  policy  to  conduct  an  AOA.5  The  AOA  is  an 
analytical  study  that  is  intended  to  compare  the  operational  effectiveness, 
cost,  and  risks  of  a  number  of  alternative  potential  solutions  to  address 
valid  needs  and  shortfalls  in  operational  capability.  The  basis  for 
conducting  an  AOA  begins  when  a  capability  need  is  validated  or  approved 
through  the  department’s  requirements  determination  process — the  Joint 
Capabilities  Integration  and  Development  System  (JCIDS)  (See  fig.  1). 


Figure  1 :  Where  AOAs  Fit  within  the  Requirements  and  Acquisition  Processes 


Requirements 


Acquisition 


CDD 


Defines  system 
capabilities 


ISA 


I 


AOA  Analysis  of  alternatives 
ICD  Initial  capabilities  document 
CDD  Capability  development  document 
MSA  Milestone  A 
MSB  Milestone  B 


Source:  GAO  presentation  of  DO D  policies. 


5  DOD’s  acquisition  policy  covering  the  time  frame  of  our  analysis,  including  guidance  for 
conducting  AOAs,  is  contained  in  DOD  Instruction  5000.2,  May  2003  revision  to  the 
Operation  of  the  Defense  Acquisition  System.  If  a  program  does  not  require  development, 
it  may  enter  at  Milestone  C.  DOD  Instruction  5000.2  was  revised  and  reissued  as  DOD 
Instruction  5000.02  in  December  2008. 
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A  sponsor,  usually  a  military  service,  submits  a  capability  proposal — called 
an  Initial  Capabilities  Document  (ICD) — through  JCIDS,  which  identifies 
the  existence  of  a  capability  gap,  the  operational  risks  associated  with  the 
gap,  and  a  recommended  solution  or  preferred  set  of  solutions  for  filling 
the  gap.  When  a  capability  proposal  is  validated,  before  a  major  defense 
acquisition  program  begins,  an  AOA  is  undertaken  to  compare  potential 
solutions  and  determine  the  most  promising  and  cost-effective  weapon 
system  to  acquire.  The  AOA  is  a  key  input  to  defining  the  system 
capabilities  of  the  major  defense  acquisition  program,  which  are 
established  in  a  capability  development  document  (CDD). 

Most  AOAs  are  sponsored  by  a  single  military  service,  but  some  may  be 
conducted  jointly  by  more  than  one  service,  in  which  case,  the  Milestone 
Decision  Authority  (MDA)  designates  a  lead  service  as  the  sponsor.  AOAs 
are  conducted  by  study  teams,  the  composition  of  which  depends  on  the 
service — most  of  the  Army’s  AOAs  are  conducted  by  the  Army’s  Training 
and  Doctrine  Command  Analysis  Center,  most  of  the  Air  Force’s  AOAs  are 
conducted  by  the  Air  Force’s  major  commands,  such  as  the  Air  Combat 
Command,  and  most  of  the  Navy’s  AOAs  are  contracted  out  to  federally 
funded  research  and  development  centers  and  the  Navy’s  various  study 
centers.  Both  the  Office  of  the  Secretary  of  Defense  (OSD)  and  the 
services  are  responsible  for  issuing  study  guidance  to  scope  the  AOA, 
which  provide  a  minimum  set  of  alternatives  to  analyze  and  shape  the 
analysis  through  a  series  of  study  questions.  Conducting  an  AOA  may  take 
anywhere  from  a  few  months  to  several  years  and  cost  from  a  few  hundred 
thousand  to  several  million  dollars  depending  on  its  scope  and  complexity. 
The  final  results  and  recommendations  of  the  AOA  are  then  presented  to 
decision  makers,  who  decide  on  which  alternative  to  select  for  program 
initiation.  According  to  the  Air  Force’s  manual6  on  conducting  AOAs,  some 
of  the  key  questions  that  decision  makers  need  the  AOA  to  answer  include: 

•  What  alternatives  provide  validated  capabilities? 

•  Are  the  alternatives  operationally  suitable  and  effective? 

•  Can  the  alternatives  be  supported? 

•  What  are  the  risks  (technical,  operational,  programmatic)  for  each 
alternative? 

•  What  are  the  life-cycle  costs  for  each  alternative? 

•  How  do  the  alternatives  compare  to  one  another? 


6  While  DOD  Instruction  S000.2  provides  general  policy  for  conducting  AOAs,  it  does  not 
specify  how  AOAs  should  be  conducted.  The  Air  Force  has  developed  detailed  guidance  for 
conducting  AOAs. 
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The  Office  of  the  Secretary  of  Defense,  Program  Analysis  and  Evaluation 
(OSD  PA&E),  plays  a  central  role  in  the  AOA  process  because  it  is 
responsible  for  providing  initial  guidance  to  the  AOA  study  team, 
reviewing  the  proposed  AOA  study  plan,  and  assessing  the  completed 
AOA.  In  carrying  out  these  functions,  OSD  PA&E  provides  a  DOD 
enterprise-level  perspective  to  AOAs  and  encourages  service  sponsors  to 
consider  all  viable  concepts  to  fill  a  capability  need,  even  if  they  were  not 
initially  considered  by  the  service  sponsors,  and  to  assess  technical  risks 
and  costs  of  each  alternative. 

The  AOA  is  one  of  several  inputs  required  for  a  program’s  initiation  at  MS 
B,* * 7  and  it  is  a  key  element  in  planning  and  establishing  a  sound  business 
case  for  a  weapon  system  program.  We  have  frequently  reported  on  the 
importance  of  using  a  solid,  executable  business  case  before  committing 
resources  to  a  new  product  development  effort.8  The  business  case  in  its 
simplest  form  is  demonstrated  evidence  that  (1)  the  warfighter’s  needs  are 
valid  and  that  they  can  best  be  met  with  the  chosen  concept,  and  (2)  the 
chosen  concept  can  be  developed  and  produced  within  existing 
resources — that  is,  proven  technologies,  design  knowledge,  adequate 
funding,  and  adequate  time  to  deliver  the  product  when  it  is  needed.  The 
AOA  addresses  the  first  point  of  a  business  case  by  providing  a  foundation 
for  developing  and  refining  the  operational  requirements  for  a  weapon 
system  program.  An  AOA  also  addresses  the  second  point  of  a  business 
case  by  providing  insight  into  the  technical  feasibility  and  costs  of 
alternatives.  By  contributing  to  business  cases,  AOAs  should  provide 
programs  with  a  sound  basis  for  program  initiation. 


'  Other  key  elements  required  for  Milestone  B  include  the  technology  readiness 

assessment,  independent  cost  estimate,  acquisition  strategy,  and  Capability  Development 

Document  (CDD). 

8  GAO,  Defense  Acquisitions:  Sound  Business  Case  Needed  to  Implement  Missile  Defense 
Agency’s  Targets  Program,  GAO-08-1113  (Washington,  D.C.:  Sept.  26,  2008),  Defense 
Acquisitions:  Improved  Business  Case  Is  Needed  for  Future  Combat  System’s  Successful 
Outcome,  GAO-06-367  (Washington,  D.C.:  Mar.  14,  2006),  and  Tactical  Aircraft:  Air  Force 
Still  Needs  Business  Case  to  Support  F/A-22  Quantities  and  Increased  Capabilities, 
GAO-05-304  (Washington,  D.C.:  Mar.  15,  2005). 
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Most  Programs  Have 
Not  Conducted  a 
Robust  Assessment  of 
Alternatives 


Most  of  the  programs  we  reviewed  either  did  not  conduct  an  AOA  or 
conducted  an  AOA  that  focused  on  a  narrow  scope  of  alternatives  and  did 
not  adequately  assess  and  compare  technical  and  other  risks  of  each 
alternative.  While  many  factors  can  affect  program  cost  and  schedule 
outcomes,  we  found  that  programs  that  conducted  a  limited  assessment  of 
alternatives  before  the  start  of  system  development  tended  to  experience 
poorer  outcomes  than  the  programs  that  conducted  more  robust  AO  As. 
According  to  several  DOD  and  program  officials,  AO  As  have  often  simply 
validated  a  concept  selected  by  the  sponsor  and  are  not  used  as  intended 
to  make  trade  offs  among  performance,  cost,  and  risks  to  achieve  an 
optimal  weapon  system  concept  that  satisfies  the  warfighter’s  needs 
within  available  resource  constraints. 


Most  Programs  Analyzed  a 
Narrow  Scope  of 
Alternatives  before  They 
Started 


Most  of  the  programs  we  reviewed  considered  a  narrow  scope  of 
alternatives  to  support  program  start.  Ten  of  the  32  programs  did  not 
conduct  AO  As  and  focused  on  an  already  selected  weapon  system 
solution.  Of  the  22  programs  that  had  AO  As,  13  of  them  examined  a  limited 
number  of  alternatives  within  a  single  weapon  system  concept  such  as 
helicopters  or  specific  classes  of  ships,  while  9  considered  a  relatively 
broad  range  of  alternatives,  by  assessing  many  alternatives  within  a  single 
weapon  concept  or  alternatives  across  multiple  concepts,  such  as 
comparing  ships  to  aircraft.  We  found  that  the  programs  that  considered  a 
broad  range  of  alternatives  tended  to  have  better  cost  and  schedule 
outcomes  than  the  programs  that  looked  at  a  narrow  scope  of  alternatives 
(see  table  1).  For  example,  1  of  the  9  programs  that  examined  a  broad  set 
of  alternatives  experienced  high  cost  or  schedule  growth  whereas  8  of  the 
13  programs  that  considered  only  a  limited  number  of  alternatives 
experienced  high  cost  or  schedule  growth. 
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Programs  That  Did  Not 
Conduct  AO  As 


Table  1 :  Comparison  of  the  Scope  of  Alternatives  Considered  with  Program  Cost 
and  Schedule  Growth 

Number  of  programs  with  cost  or 
schedule  growth’ 

Scope  of  alternatives'5 

Low 

Moderate  High 

No  AOA  conducted 

7 

0  3 

AOA  included  broad  scope  of  alternatives 

7 

1  1 

AOA  included  narrow  scope  of  alternatives 

4 

1  8 

Source:  GAO. 


a  Cost  growth:  High  =  25  percent  or  greater  growth  in  development  cost  (or  procurement  costs  for 
nondevelopmental  programs)  from  initial  baseline  to  current  estimates,  Moderate  =  10  to  24  percent 
growth  in  development  cost  (or  procurement  costs  for  nondevelopmental  programs)  from  initial 
baseline  to  current  estimates,  Low  =  less  than  10  percent  growth  in  development  cost  (or 
procurement  costs  for  nondevelopmental  programs)  from  initial  baseline  to  current  estimates. 

Schedule  growth:  High  =  greater  than  12  month  delay  for  the  initial  operational  capability  date  or 
acquisition  cycle,  Moderate  =7  to  12  month  delay  for  the  initial  operational  capability  date  or 
acquisition  cycle,  Low  =  less  than  7  month  delay  for  the  initial  operational  capability  date  or 
acquisition  cycle. 

b  Narrow  scope  of  alternatives  =2-5  alternatives  within  one  concept;  broad  scope  of  alternatives  =8- 
26  alternatives  within  one  concept,  or  alternatives  within  multiple  concepts. 


For  various  reasons,  10  of  the  32  weapon  programs  we  reviewed  did  not 
have  formal  AOAs  to  support  program  start  (see  table  2).  For  7  of  these 
programs,  it  may  have  been  appropriate  not  to  conduct  the  AOA  because 
the  programs  involved  a  planned  modernization  to  an  existing  weapon 
system  or  there  was  support  from  other  analyses  to  warrant  the  chosen 
concept.  This  was  the  case,  for  example,  with  the  Navy’s  Standard  Missile 
6  (SM-6)  program.  Because  the  missile  was  the  next  planned  increment  in 
a  long  history  of  missile  development  efforts  and  an  AOA  had  been 
conducted  for  the  previous  standard  missile  increment,  a  separate  AOA  for 
SM-6  was  considered  repetitive  and  waived.  The  program  started 
development  in  2004  and  has  remained  on  track  with  its  planned  cost  and 
schedule  objectives.  Similarly,  an  AOA  was  not  conducted  for  the  Air 
Force’s  Global  Positioning  System  IIIA  program  because  there  was  a  body 
of  analysis  available  that  served  the  purpose  of  an  AOA  and  the  proposed 
program  was  considered  a  follow-on  increment  to  a  multiprogram  effort  to 
modernize  global  positioning  system  capabilities.  Since  it  started 
development  in  2008,  the  program  has  remained  on  cost  and  schedule. 
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Table  2:  Programs  That  Did  Not  Conduct  AOAs 

Program 

Reasons  for  not  conducting  an  AOA 

Cost  or  schedule  growth 
(low,  moderate,  or  high)3 

B-2  RMP  -  B-2  Radar  Modernization 
Program 

The  Radar  Modernization  Program  (RMP)  only  changes  the 
operating  frequency  of  the  radar  with  no  functional  upgrades. 
The  B-2  system  program  office  considers  the  analysis  leading 
to  the  radar  modernization  to  be  an  informal  AOA. 

Low 

B-2  EHF  INCREMENT  1  -  B-2 
Advanced  Extremely  High  Frequency 
SatCom  Capability 

The  program  had  previous  analyses  on  processors  and 
antennas  that  fulfilled  the  intent  of  an  AOA.  The  milestone 
decision  authority  for  the  program  waived  the  AOA 
requirement. 

Low 

F-22A  Modernization  Program 

The  F -22  program  had  completed  a  costand  operational 
effectiveness  analysis  in  1991.  No  analysis  was  conducted  for 
the  modernization  component  of  the  F -22  program. 

High 

Global  Positioning  Satellite  III  (GPS 
IMA) 

The  Air  Force  had  conducted  multiple  studies  and  analyses 
that  together  fulfilled  the  intentof  an  AOA. 

Low 

JTRS  HMS  -  J  oint T actical  Radio 
System  Handheld,  Manpack,  and 
Small  Form  Fit  Radios 

The  Army  conducted  a  study  which  represents  the  results  of  a 
mission  needs  analysis,  an  operational  requirements  analysis, 
and  a  cost-effectiveness  and  affordability  analysis.  There  was 
no  comparison  of  alternatives.  This  report  served  as  their 

AOA. 

High 

MP  RTIP  -  Multi-Platform  Radar 
Technology  Insertion  Program 

The  RadarTechnology  Insertion  Program  was  initially  a 
Previously  Planned  P  roduct  Improvement  for  the  J  ointSTARS 
aircraft. 

Low 

NMT  -  Advanced  Extremely  High 
Frequency  Navy  Multiband  Terminal 
Satellite  Program 

This  NMT  program  began  as  an  ACAT  II,  and  ACAT  II 
programs  are  not  required  to  have  AOAs  at  program  start. 

Low 

SBSS  BIO  -  Space-Based  Space 
Surveillance  Block  10 

An  AOA  was  not  conducted  because  of  the  Air  Force's 
direction  to  quickly  develop  and  field  a  replacement  for  the 
current  satellite  surveillance  system  in  order  to  prevent  an 
operational  capability  gap. 

Low 

SKY  WARRIOR  -  Unmanned  Aircraft 
System 

The  Army  had  already  made  the  decision  to  procure  an 
unmanned  aerial  system.  The  Army  argued,  among  other 
things,  that  the  source  selection  process  would  drive 
competitive  selection  of  alternatives. 

High 

SM-6  -  Standard  Missile-6 

No  AOA  was  conducted  because  several  analyses  were 
completed  for  previous  iterations  of  the  program. 

Low 

Source:  GAO  analysis  of  data  provided  by  DOD  program  offices. 


a  Cost  growth:  High  =  25  percent  or  greater  growth  in  development  cost  (or  procurement  costs  for 
nondevelopmental  programs)  from  initial  baseline  to  current  estimates,  Moderate  =  10  to  24  percent 
growth  in  development  cost  (or  procurement  costs  for  nondevelopmental  programs)  from  initial 
baseline  to  current  estimates,  Low  =  less  than  10  percent  growth  in  development  cost  (or 
procurement  costs  for  nondevelopmental  programs)  from  initial  baseline  to  current  estimates. 

Schedule  growth:  High  =  greater  than  12  month  delay  for  the  initial  operational  capability  date  or 
acquisition  cycle,  Moderate  =7  to  12  month  delay  for  the  initial  operational  capability  date  or 
acquisition  cycle,  Low  =  less  than  7  month  delay  for  the  initial  operational  capability  date  or 
acquisition  cycle. 
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Programs  That  Conducted 
AOAs 


However,  in  the  other  3  programs  that  did  not  have  AOAs,  the 
requirements  and  development  effort  proved  to  be  more  demanding  and 
cost  and  schedule  growth  occurred.  In  the  case  of  the  Army’s  Sky  Warrior 
Unmanned  Aerial  System  (UAS)  program,  an  Army  executive  waived  the 
AOA  requirement  because  the  Army  believed,  among  other  things,  that  the 
source  selection  process  would  provide  an  adequate  way  to  compare 
alternatives.  However,  when  the  Air  Force  and  Joint  Staff  were  reviewing 
the  Sky  Warrior’s  draft  requirements  and  acquisition  documentation,  they 
raised  concerns  that  the  requirements  potentially  duplicated  capability 
provided  by  the  Air  Force’s  Predator  UAS.  The  Army  cited  the  urgent  need 
of  battlefield  commanders  for  the  capability  and  gained  approval  to 
proceed  to  source  selection.  Three  years  after  the  Sky  Warrior  AOA  was 
waived,  the  Deputy  Secretary  of  Defense  directed  that  the  two  UAS 
programs  be  combined  into  a  single  acquisition  program  to  achieve 
efficiencies  in  areas  such  as  common  development,  procurement,  and 
training  activities.  However,  the  Army  and  Air  Force  have  continued  to 
pursue  unique  systems.  In  the  meantime,  the  Sky  Warrior  UAS  has 
experienced  a  138  percent  increase  in  total  cost  and  47-month  schedule 
delay  from  original  plans.  By  relying  on  industry-provided  information  in 
source  selection  and  not  conducting  an  independent  AOA,  the  Army 
missed  an  opportunity  to  gain  a  better  understanding  of  the  other  services’ 
UAS  capabilities,  and  pursue  an  acquisition  strategy  that  would  have  taken 
advantage  of  commonalities  and  used  resources  more  efficiently. 

Of  the  22  programs  that  conducted  AOAs,  13  focused  on  a  limited  number 
of  alternatives  within  a  single  weapon  system  concept  while  9  focused  on 
many  alternatives9  (see  table  3).  According  to  DOD  and  service  officials, 
the  scope  of  an  AOA  can  be  different  for  each  program  and  dependent 
upon  many  factors,  including  the  nature  of  the  capability  need,  the 
proposed  time  frame  for  fielding  the  capability,  and  the  type  of  program 
being  pursued-whether  it  is  a  new  development  start,  a  modification  of  a 
commercially  available  system,  or  an  upgrade  to  an  existing  system.  As  a 
result,  AOAs  that  focus  on  a  limited  number  of  alternatives  within  a  single 
weapon  system  concept  may  be  appropriate  in  some  cases.  For  instance, 


9  Although  DOD  has  no  specific  criteria  for  how  many  alternatives  are  to  be  considered  in 
an  AOA,  Air  Force  guidance  indicates  that  AOAs  should  consider  a  comprehensive  set  of 
alternatives  representing  all  reasonable  solutions.  For  the  AOAs  we  reviewed,  we 
categorized  “limited”  number  of  alternatives  as  AOAs  that  examined  two  to  five 
alternatives  within  a  single  weapon  system  concept  and  “many”  alternatives  as  AOAs  that 
examined  eight  or  more  alternatives  within  a  single  weapon  system  concept  or  alternatives 
across  multiple  weapon  system  concepts. 
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when  the  capability  need  was  defined  in  terms  of  upgrading  an  existing 
weapon  system,  AO  As  focused  on  refining  a  single  platform  concept  and 
its  system-level  specifications  and  attributes.  The  AOA  for  the  Army’s 
Apache  Block  III  program  is  an  example  of  an  appropriately,  but  narrowly 
scoped  AOA.  It  examined  various  block  upgrade  options  for  the  existing 
Longbow  Apache  helicopter  to  improve  interoperability  and  other 
shortcomings  in  the  helicopter.  The  program  started  development  in  2006 
and  has  remained  on  track  with  its  planned  cost  and  schedule  objectives. 


Table  3:  Scope  of  Alternatives  Considered  in  AOAs 

AOAs  with  narrow  scope  of 
alternatives" 

Cost  or  schedule  growth 
(low,  moderate,  or  high)" 

AOAs  with  broad  scope  of 
alternatives" 

Cost  or  schedule  growth 
(low,  moderate,  or  high)b 

AGM-88E  AARGM  -  AGM-88E 
Advanced  Anti-Radiation  Guided 
Missile  (AARGM)  Program 

Low 

BAMS  -  Broad  Area  Maritime 
Surveillance 

High 

AB3  -  Apache  Block  III 

Low 

EA-18G  -  Electronic  attack  variant 
of  the  F/A-18  aircraft 

Low 

ARH  -  Armed  Reconnaissance 
Helicopter 

High 

CH-53K-  Heavy  Lift  Replacement 
Program 

Low 

CVN  21  -  Next  Generation 

Nuclear  Aircraft  Carrier 

Moderate 

J  CA  -  J  oint  Cargo  Aircraft 

Low 

DDG  1000  -  ZUMWALT  CLASS 
Destroyer 

High 

J  LENS  -  J  oint  Land  Attack  Cruise 
Missile  Defense  Elevated  Netted 
Sensor  System 

Low 

E-2D  AHE  -  E-2D  Advanced 
Hawkeye 

High 

LUH  -  Light  Utility  Helicopter 

Low 

FCS  -  Future  Combat  Systems 

High 

MUOS  -  Mobile  User  Objective 
System 

Moderate 

JTRS  AMF  -  J  oint T actical  Radio 
System  Airborne,  Maritime/Fixed 
Station 

Low 

P-8A  -  Poseidon  Program 

Low 

LCS  -  Littoral  Combat  Ship 

High 

SDB  1  -  Small  Diameter  Bomb 
Increment  1 

Low 

LHA  REPLACEMENT  -  New 
Amphibious  AssaultShip 

Low 

MPS  -  Mission  Planning  System 

High 

VH-71  -  Presidential  Helicopter 
Fleet  Replacement  Program 

High 

WIN-T  -  Warfighter  Information 
Network  -  Tactical 

High 

Source:  GAO  analysis  of  data  provided  by  DOD  program  offices. 


3  Narrow  scope  of  alternatives  =  2-5  alternatives  within  one  concept;  broad  scope  of  alternatives  =  8- 
26  alternatives  within  one  concept,  or  alternatives  within  multiple  concepts. 
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b  Cost  growth:  High  =  25  percent  or  greater  growth  in  development  cost  (or  procurement  costs  for 
nondevelopmental  programs)  from  initial  baseline  to  current  estimates,  Moderate  =  10  to  24  percent 
growth  in  development  cost  (or  procurement  costs  for  nondevelopmental  programs)  from  initial 
baseline  to  current  estimates,  Low  =  less  than  10  percent  growth  in  development  cost  (or 
procurement  costs  for  nondevelopmental  programs)  from  initial  baseline  to  current  estimates. 

Schedule  growth:  High  =  greater  than  12  month  delay  for  the  initial  operational  capability  date  or 
acquisition  cycle,  Moderate  =7  to  12  month  delay  for  the  initial  operational  capability  date  or 
acquisition  cycle,  Low  =  less  than  7  month  delay  for  the  initial  operational  capability  date  or 
acquisition  cycle. 

In  a  few  of  the  other  AO  As  that  had  a  narrow  scope,  the  capability  need 
involved  the  replacement  of  an  aging  weapon  system  and  the  AOAs 
presumed  that  the  concept  of  the  aging  weapon  system  was  the 
appropriate  starting  point  for  analysis  rather  than  examining  whether 
other  concepts  could  also  meet  the  need.  For  example,  the  AOA  for  the 
Army’s  Armed  Reconnaissance  Helicopter  (ARH)  program,  which  was 
intended  to  replace  the  aging  Kiowa  helicopter  fleet  and  improve  attack 
and  reconnaissance  capabilities,  examined  two  options:  improving  the 
legacy  Kiowa  helicopter  or  procuring  nondevelopmental  helicopters.  The 
AOA  did  not  explore  other  potential  solutions,  such  as  developing 
unmanned  aerial  systems,  increasing  the  purchase  of  existing  attack 
helicopters,  increasing  the  purchase  of  other  reconnaissance  assets,  or 
relying  on  a  mix  of  solutions.  After  3  years  of  development,  the  ARH 
program’s  research  and  development  costs  increased  from  about  $360 
million  to  $940  million.  A  Center  for  Naval  Analyses  report  commissioned 
by  the  Army  after  the  ARH  program  began  having  execution  problems 
identified  several  factors  that  contributed  to  the  significant  cost  growth, 
including  questionable  requirements,  an  aggressive  schedule,  limited 
oversight,  and  a  perceived  preference  for  one  helicopter  model.  As  a  result 
of  the  cost  growth  and  other  problems,  DOD  cancelled  the  program  in 
2008  after  determining  that  at  least  one  alternative  could  provide  equal  or 
greater  capability  at  less  cost. 

Most  of  the  programs  (7  of  9)  that  examined  a  broad  scope  of  alternatives 
have  tracked  well  with  their  planned  cost  and  schedule  targets.  The  AOA 
for  the  Navy’s  P-8A  Multi-mission  Maritime  Aircraft,  which  is  a  program 
designed  to  replace  the  P-3C  aircraft  and  provide  maritime  patrol  and 
reconnaissance  for  the  Navy,  explored  multiple  concepts  and  many 
alternatives  in  response  to  study  guidance  issued  by  OSD  PA&E,  including 
several  nonmanned  aircraft  alternatives  such  as  submarines,  helicopters, 
and  UAS.  The  AOA  concluded  that  a  manned  aircraft  would  still  be  the 
best  option  to  replace  the  P-3C.  However,  the  AOA  also  helped  the  Navy  to 
recognize  that  a  UAS  could  perform  some  of  the  maritime  patrol  missions 
as  an  adjunct  platform,  eventually  leading  to  the  Broad  Area  Maritime 
Surveillance  (BAMS)  UAS  AOA  and  program.  The  P-8A  program  has  not 
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experienced  cost  growth  over  its  4  years  of  development  and  remains  on 
schedule.  Similarly,  the  Joint  Land  Attack  Cruise  Missile  Defense  Elevated 
Netted  Sensor  System  (JLENS),  which  is  designed  to  provide  over  the 
horizon  detecting  and  tracking  of  land  attack  cruise  missile  and  other 
targets,  had  an  AOA  that  explored  alternatives  across  multiple  concepts, 
including  aerostat  sensors,  sea-based  sensors,  and  nonaerostat  elevated 
sensors.  The  Army  chose  the  aerostat  concept  and  has  developed  an 
incremental  program  that  has  experienced  low  cost  and  schedule  growth 
since  starting  development  in  2005. 


Many  AO  As  Have  Not 
Adequately  Assessed  Risks 
for  the  Alternatives 


DOD  acquisition  policy  requires  that  AO  As  assess  the  technical  risk  of 
alternatives,  but  it  does  not  provide  criteria  and  guidance  for  how  and  to 
what  extent  technical  risks  should  be  addressed  and  it  does  not  specify 
that  other  types  of  risks  should  be  assessed.10  Risks  are  important  to 
assess  because  there  may  be  technical,  programmatic,  or  operational 
uncertainties  associated  with  different  alternatives  that  should  be 
considered  in  determining  the  best  weapon  system  approach.  For 
example,  it  may  be  the  case  that  one  alternative  is  more  effective  than 
another  in  meeting  a  capability  need  but  has  more  technical  or  other  risks 
that  may  make  the  alternative  infeasible  to  develop.  Many  of  the  AO  As  we 
reviewed  (12  of  the  22)  conducted  limited  assessments  of  the  risks  of  each 
alternative  presented  (see  table  4). 11  Some  AOAs  we  reviewed  did  not 
examine  risks  at  all,  focusing  only  on  the  operational  effectiveness  and 
costs  of  alternatives.  Other  AOAs  had  relatively  limited  risk  assessments. 
For  example,  several  AOAs  did  not  discuss  integration  risks  even  though 
they  were  examining  modified  commercial  systems  that  required  the 
integration  of  subsystems  or  equipment  packages,  while  other  AOAs  did 
not  examine  the  schedule  risks  of  the  various  alternatives,  despite 
accelerated  schedules  and  fielding  dates  for  the  programs.  We  found  that 
programs  with  AOAs  that  conducted  a  more  comprehensive  assessment  of 


10  While  DOD  acquisition  policy  (DODI  S000.2)  indicates  that  an  AOA  shall  assess 
technology  risk  and  maturity,  Air  Force  guidance  for  conducting  AOAs  advocates  that 
programmatic  and  operational  risks  should  be  considered  as  well.  The  Air  Force  guidance 
indicates  that  AOA  study  teams  should  determine,  as  a  first  step  in  conducting  a  risk 
analysis,  what  technical,  programmatic,  and  operational  factors  are  relevant  to  each 
alternative.  For  example,  open  architecture,  schedule,  overarching  dependencies,  and 
maintenance  requirements  may  be  important  risk  factors  for  some  AOAs. 

11  We  defined  “limited”  risk  assessments  as  AOAs  that  did  not  examine  risks  at  all  or 
examined  only  one  risk  factor  such  as  technology  maturity.  We  defined  AOAs  as  having  a 
more  robust  risk  assessment  if  they  examined  multiple  risk  factors  such  as  technology, 
integration,  and  schedule. 
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risks  tended  to  have  better  cost  and  schedule  outcomes  than  those  that  did 
not  (see  table  5). 

Table  4:  Risk  Assessment  of  AOAs  by  Program 

AOAs  with  no  or  limited 
assessment  of  risks9 

Cost  or  schedule  growth 
(low,  moderate,  or  high)" 

AOAs  with  adequate 
assessment  of  risks9 

Cost  or  schedule  growth 
(low,  moderate,  or  high)" 

AGM-88E  AARGM  -  AGM  88E 
Advanced  Anti- Radiation  Guided 
Missile  (AARGM)  Program 

Low 

BAMS  -  Broad  Area  Maritime 
Surveillance 

High 

AB3  -  Apache  Block  III 

Low 

DDG  1000  -  ZUM WALT  CLASS 
Destroyer 

High 

ARH  -  Armed  Reconnaissance 
Helicopter 

High 

EA-18G  -  Electronic  attack  variant 
of  the  F/A-18  aircraft 

Low 

CVN-21  -  Next  Generation 

Nuclear  Aircraft  Carrier 

Moderate 

CH-53K-  Heavy  Lift  Replacement 
Program 

Low 

E-2D  AHE  -  E-2D  Advanced 
Hawkeye 

High 

J  CA  -  J  oint  Cargo  Aircraft 

Low 

FCS  -  Future  Combat  Systems 

High 

J  TRS  AMF  -  J  oint  Tactical  Radio 
System  Airborne  &  Maritime/Fixed 
Station 

Low 

J  LENS  -  J  ointLand  Attack  Cruise 
Missile  Defense  Elevated  Netted 
Sensor  System 

Low 

LUH  -  Light  Utility  Helicopter 

Low 

LCS  -  Littoral  Combat  Ship 

High 

MUOS  -  Mobile  User  Objective 
System 

Moderate 

LHA  REPLACEMENT  -  New 
Amphibious  AssaultShip 

Low 

P-8A  -  Poseidon  Program 

Low 

MPS  -  Mission  Planning  System 

High 

SDB  1  -  Small  Diameter  Bomb 
Increment  1 

Low 

VH-71  -  Presidential  Helicopter 
Fleet  Replacement  Program 

High 

WIN-T  -  Warfighter  Information 
Network  -  Tactical 

High 

Source:  GAO  analysis  of  data  provided  by  DOD  program  offices. 


Page  13 


GAO-09-665  Analysis  of  Alternatives 


a  No  assessment  =  no  discussion  in  AOA  of  technical,  integration,  or  other  risks;  limited  assessment  = 
only  technical  risk  addressed  or  not  all  of  the  alternatives  are  assessed  for  technical  and  other  risks; 
adequate  assessment  =  all  alternatives  are  assessed  for  multiple  factors  such  as  technical, 
integration,  and  other  risks. 

b  Cost  growth:  High  =  25  percent  or  greater  growth  in  development  cost  (or  procurement  costs  for 
nondevelopmental  programs)  from  initial  baseline  to  current  estimates,  Moderate  =  10  to  24  percent 
growth  in  development  cost  (or  procurement  costs  for  nondevelopmental  programs)  from  initial 
baseline  to  current  estimates,  Low  =  less  than  10  percent  growth  in  development  cost  (or 
procurement  costs  for  nondevelopmental  programs)  from  initial  baseline  to  current  estimates. 

Schedule  growth:  High  =  greater  than  12  month  delay  for  the  initial  operational  capability  date  or 
acquisition  cycle,  Moderate  =  7  to  12  month  delay  for  the  initial  operational  capability  date  or 
acquisition  cycle,  Low  =  less  than  7  month  delay  for  the  initial  operational  capability  date  or 
acquisition  cycle. 


Table  5:  Comparison  of  Quality  of  AOA  Risk  Assessments  and  Program  Cost  and 
Schedule  Growth 

Number  of  programs  with  cost  or 
schedule  growth8 

Quality  of  risk  assessment" 

Low  Moderate  High 

AO  As  with  no  or  limited  assessment  of  risks 

4  17 

AO  As  with  adequate  assessment  of  risks 

7  12 

Source:  GAO. 


a  Cost  growth:  High  =  25  percent  or  greater  growth  in  development  cost  (or  procurement  costs  for 
nondevelopmental  programs)  from  initial  baseline  to  current  estimates,  Moderate  =  10  to  24  percent 
growth  in  development  cost  (or  procurement  costs  for  nondevelopmental  programs)  from  initial 
baseline  to  current  estimates,  Low  =  less  than  10  percent  growth  in  development  cost  (or 
procurement  costs  for  nondevelopmental  programs)  from  initial  baseline  to  current  estimates. 

Schedule  growth:  High  =  greater  than  12  month  delay  for  the  initial  operational  capability  date  or 
acquisition  cycle,  Moderate  =  7  to  12  month  delay  for  the  initial  operational  capability  date  or 
acquisition  cycle,  Low  =  less  than  7  month  delay  for  the  initial  operational  capability  date  or 
acquisition  cycle. 

b  No  assessment  =  no  discussion  in  AOA  of  technical,  integration,  or  other  risks;  limited  assessment  = 
only  technical  risk  addressed  or  not  all  of  the  alternatives  are  assessed  for  technical  and  other  risks; 
adequate  assessment  =  all  alternatives  are  assessed  for  multiple  factors  such  as  technical, 
integration,  and  other  risks. 


AO  As  that  do  not  examine  risks  could  provide  overly  optimistic 
assessments  of  alternatives,  which  do  not  provide  for  sound  business  case 
decisions.  Comparing  risks  across  alternatives  is  especially  critical  for  new 
development  programs,  which  rely  on  breakthrough  technologies  and 
assume  that  technology  will  be  achieved  as  planned.  Of  the  22  programs 
that  had  AO  As,  8  were  new  development  starts  involving  technology 
development.  Of  the  8  new  development  starts,  only  4  had  AO  As  that 
performed  adequate  risk  analyses.  The  other  4  AO  As  did  not  assess 
technical,  integration,  or  other  risks  as  criteria  for  comparing  the 
alternatives  or  neglected  to  analyze  these  risks  altogether.  For  example, 
the  AOA  for  the  Future  Combat  Systems  (FCS),  one  of  most  complex  and 
technically  challenging  programs  ever  undertaken  according  to  the  Army, 
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assessed  the  technical  risks  of  each  of  the  new  development  concepts  for 
FCS,  but  did  not  assess  and  compare  the  risks  with  those  of  the  other 
alternatives.  The  AOA  concluded  that  the  new  FCS  development  option 
was  more  costly  but  more  operationally  effective  than  the  baseline  and 
improved  baseline  alternatives.12  By  not  comparing  the  risks  of  the 
alternatives,  the  FCS  AOA  missed  an  opportunity  to  provide  the  Army  with 
a  meaningful  trade  off  among  operational  effectiveness,  costs,  and  risks. 
Now,  after  6  years  of  development,  some  of  the  critical  technologies  for 
the  FCS  program  are  still  immature.  The  latest  estimates  for  the  program 
show  that  development  costs  have  grown  38  percent  or  about  $8  billion, 
and  the  fielding  date  has  been  delayed  57  months.  As  a  result,  DOD 
recently  proposed  canceling  the  FCS  acquisition  program. 

Also,  the  AOA  for  the  Army’s  Warfighter  Information  Network-Tactical 
(WIN-T)  program,  which  involves  development  of  new  on-the-move 
networking  capabilities,  did  not  address  technical  or  programmatic  risks. 
Army  officials  stated  that  WIN-T  was  largely  based  on  a  concept  that  did 
not  have  well-defined  requirements  of  the  proposed  network  and 
operations,  and  the  WIN-T  development  alternative  in  the  AOA  was  based 
on  preliminary  design  concepts,  from  two  competing  contractors,  which 
were  blended  together  by  the  Army.  The  AOA  did  not  take  these  risks  into 
account  and  concluded  that  the  new  WIN-T  alternative  was  the  most 
operational  and  cost-effective  solution  available.  In  March  of  2007,  the 
WIN-T  program  had  a  Nunn  McCurdy  cost  breach  (25  percent  or  more  unit 
cost  growth)  and  was  subsequently  restructured  by  DOD.  Insufficient 
technical  readiness  was  cited  as  one  of  the  key  factors  leading  to  the  cost 
breach. 

Assessing  risks  is  also  important  for  programs  based  on  commercial 
products  that  require  significant  modifications.  Based  upon  a  recent 
Defense  Science  Board  report  on  buying  commercially-based  defense 
systems,  programs  that  do  not  assess  the  systems  engineering  and 
programmatic  risks  of  alternatives  do  not  understand  the  true  costs 
associated  with  militarizing  commercial  platforms  or  integrating  various 


12  The  AOA  for  the  FCS  program  examined  the  following  seven  alternatives:  Base  Case  - 
Heavy  Force,  Base  Case  -  Light  Force,  Base  Case  -  Interim  Force,  FCS  -  Block  II,  FCS  - 
Block  I,  FCS  -  Increment  1,  Product  Improved  Stryker  Brigade  Combat  Team. 
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commercial  components.13  As  a  result  of  this  incomplete  understanding  of 
inherent  technical  and  integration  risks  of  programs,  DOD  fails  to  fully 
take  advantage  of  efficiencies  and  cost  savings  from  commercially 
available  technologies.  Several  of  the  programs  we  reviewed  that  involved 
modified  commercial  products  had  AO  As  with  weak  risk  assessments.  For 
example,  the  AOA  for  the  Marine  Corps’  replacement  for  the  Presidential 
Helicopter,  VH-71,  failed  to  assess  the  technical,  integration,  and  schedule 
risks  associated  with  its  three  alternatives.  It  instead  compared 
alternatives  based  on  costs  and  performance  attributes,  such  as  cabin  size, 
deployability,  and  performance.  One  program  official  stated  that  the  focus 
of  the  VH-71  AOA  was  to  merely  identify  platforms  that  had  the  best 
probability  of  meeting  the  requirements.  According  to  a  statement  by  the 
Secretary  of  Defense,  the  program’s  costs  have  nearly  doubled,  increasing 
from  $6.5  billion  to  $13  billion,  and  the  schedule  has  fallen  behind  by 
several  years.  DOD  recently  cancelled  the  program.  The  Defense  Science 
Board,  which  assessed  the  VH-71  program,  concluded  that  some  of  the 
program’s  requirements  plainly  exceeded  the  limits  of  the  available 
technology  and  schedule. 


Choosing  an 
Alternative  Too  Early 
and  Conducting  AO  As 
under  Compressed 
Time  Frames  and 
without  Effective 
Guidance  Limit  the 
Scope  and  Quality  of 
AOAs 


We  identified  several  factors  that  may  have  limited  the  effectiveness  of 
AOAs  and  their  ability  to  identify  the  most  promising  option  and 
contribute  to  a  sound  business  case  for  starting  a  weapon  system  program: 
(1)  service  sponsors  lock  into  a  solution  early  on  when  a  capability  need  is 
first  validated  through  DOD’s  requirements  process  and  before  an  AOA  is 
conducted;  (2)  AOAs  are  conducted  under  compressed  time  frames  in 
order  to  meet  a  planned  milestone  review  or  fielding  date  and  their  results 
come  too  late  to  inform  key  trade  off  decisions;  and  (3)  DOD  does  not 
always  provide  guidance  for  conducting  individual  AOAs.  The  AOAs  with 
one  or  more  of  these  factors  tended  to  be  AOAs  that  had  a  limited  scope 
and  assessment  of  risks  (see  table  6). 


13  Defense  Science  Board,  Buying  Commercial:  Gaining  the  Cost/Schedule  Benefits  for 
Defense  Systems.  Report  of  the  Defense  Science  Board  Task  Force  on  Integrating 
Commercial  Systems  into  the  DOD ,  Effectively  and  Efficiently  (Washington,  D.C.: 
February  2009). 
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Table  6:  Factors  Associated  with  AOAs  That  Had  Different  Scope  and  Risk  Assessments 


Choosing  an  alternative  too 
early 

Problematic  timing 

No  or  late  PA&E  guidance 

AOAs  with  limited 
scope  and  risk 
assessments 

ARH  -  Armed 

Reconnaissance  Helicopter 
CVN  21  -  Next  Generation 
Nuclear  Aircraft  Carrier 

FCS  -  Future  Combat 
Systems 

LCS  -  Littoral  Combat  Ship 

AB3  -  Apache  Block  III 

ARH  -  Armed  Reconnaissance 
Helicopter 

CVN  21  -  Next  Generation  Nuclear 
Aircraft  Carrier 

FCS  -  Future  Combat  Systems 

LCS  -  Littoral  Combat  Ship 

VH-71  -  Presidential  Helicopter  Fleet 
Replacement  Program 

WIN-T  -  Warfighter  Information 
Network-Tactical 

AGM-88E  AARGM  -  AGM-88E 
Advanced  Anti-Radiation  Guided 
Missile  (AARGM)  Program 

E-2D  AHE  -  E-2D  Advanced 
Hawkeye 

FCS  -  Future  Combat  Systems 
LCS  -  Littoral  Combat  Ship 

MPS  -  Mission  Planning  System 
VH-71  -  Presidential  Helicopter 
Fleet  Replacement  Program 

AOAs  with  broad  scope  CH-53K  -  Heavy  Lift  CH-53K  -  Heavy  Lift  Replacement 

and  adequate  risk  Replacement  Program  Program 

assessments  LUH  -  Light  Utility  Helicopter  LUH  -  Light  Utility  Helicopter 


BAMS  -  Broad  Area  Maritime 
Surveillance 

CH-53K  -  Heavy  Lift  Replacement 
Program 

SDB  I  -  Small  Diameter  Bomb 
Increment  I 


Source:  GAO  analysis  of  data  provided  by  DOD  program  offices. 


Service  Sponsors  Lock 
Into  a  Program  Solution  or 
Establish  Requirements 
Prior  to  the  AOA 


In  developing  a  capability  proposal,  sponsors  not  only  justify  the  need  to 
fill  an  existing  capability  gap,  but  also  conduct  an  assessment — called  a 
functional  solutions  analysis  (FSA) — to  identify  a  potential  concept  or  set 
of  solutions  to  fill  the  gap.  The  identification  of  a  potential  concept  is 
intended  to  provide  a  general  approach  for  addressing  the  gap  and  set  the 
stage  for  a  more  in-depth  assessment  of  alternatives  to  be  conducted  in  the 
AOA.  In  four  cases,  AOAs  were  limited  because  program  sponsors  had 
decided  on  a  preferred  solution  prior  to  the  AOA,  when  a  capability  need 
was  first  proposed  through  the  department’s  requirements  determination 
process.  Approval  of  the  capability  proposal  then  led  to  a  narrowly  scoped 
AOA  that  supported  or  refined  the  preferred  solution.  According  to  DOD 
officials,  the  analysis  supporting  a  capability  proposal  is  generally 
conducted  by  the  operational  requirements  community  within  a  military 
service  and  contains  only  rudimentary  assessments  of  the  costs  and 
technical  feasibility  of  the  solutions  identified. 


With  the  Armed  Reconnaissance  Helicopter  program,  for  example,  the 
Army  proposed  acquiring  an  armed  reconnaissance  helicopter  after  the 
termination  of  the  Comanche  helicopter  program,  which  had  experienced 
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significant  cost  and  schedule  problems.14  While  the  initial  capability 
proposal  submitted  to  JCIDS  for  the  ARH  considered  nonhelicopter 
concepts,  such  as  unmanned  aerial  systems,  the  Army  concluded  that  a 
modified  version  of  an  existing  armed  reconnaissance  helicopter  was  the 
preferred  solution.  According  to  Army  officials,  the  modified  helicopter 
solution  was  pushed  in  part  because  there  was  a  desire  to  field  a  system 
within  a  relatively  short  time  frame,  a  similar  helicopter  variant  was  in  use 
by  the  special  operations  forces,  and  funding  available  from  the 
terminated  Comanche  helicopter  program  needed  to  be  used  quickly. 
Because  the  Army  effectively  locked  into  a  solution  in  this  early  stage,  the 
AOA  primarily  focused  on  comparing  the  performance  and  costs  of 
existing  helicopter  alternatives  (see  fig.  2). 


14  The  termination  of  the  Comanche  helicopter  program  provided  the  Army  the  opportunity 
to  redirect  its  resources  to  fund  a  broad  range  of  programs  and  activities,  including  the 
Armed  Reconnaissance  Helicopter,  which  is  intended  to  address  numerous  shortcomings 
in  the  reconnaissance  helicopter  fleet. 
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Figure  2:  Alternatives  Considered  for  the  ARH  Program 


Alternatives  Considered 
Before  AOA 

(in  Initial  Capabilities  Document) 


Upgrade  of  existing  ARH 

Increase  use  of  attack  helicopters 

Initiate  a  new-start  ARH  program 

Increased  use  of  Reconnaissance 
Surveillance  Target  Acquisition  assets 

Unmanned  CombatArmed  Rotorcraft 

Unmanned  Aerial  Vehicle  Systems 


Armed  Reconnaissance  Helicopter  (ARH) 

Alternatives  Examined  in  AOA 


Platform  Chosen 


Upgrade  of  existing  ARH  alternatives: 

•  Improve  Kiowa  (legacy  helicopter) 

•  Non-developmental  helicopters 

1)  OH  6/MH-6  MELB 

2)  Eurocopter  EC -635 

3)  Eurocopter  AS-550  C3 

4)  Mil  Bell  407 


Mil  Bell  407 
helicopter 


Upgrade  of  existing  ARH  is  preferred  solution 


Source:  GAO  presentation  of  DOD  data. 


Similarly,  we  have  previously  reported15  that  the  Navy  began  the  Littoral 
Combat  Ship  (LCS)  program  before  fully  examining  alternatives. 

Beginning  in  1998,  the  Navy  conducted  a  series  of  wargames  and  studies  to 
test  new  concepts  for  surface  combatant  ships  that  could  address  known 
threats  in  littoral  areas.  Following  these  efforts,  the  Navy  began  an 
analysis  of  multiple  concepts  study16  in  2002  to  further  refine  the  Navy’s 
preferred  solution — a  new  warship  along  the  lines  of  LCS.  Concurrently, 
the  Navy  established  an  LCS  program  office  and  issued  a  request  for 


15  GAO,  Defense  Acquisitions:  Plans  Need  to  Allow  Enough  Time  to  Demonstrate 
Capability  of  First  Littoral  Combat  Ships,  GAO  05-255  (Washington,  D.C.:  Mar.  1,  2005). 

16  The  Navy  conducted  an  Analysis  of  Multiple  Concepts,  one  of  the  studies  required  by 
DOD  acquisition  policy  in  2002  for  program  initiation,  which  was  subsequently  approved  by 
OSD  PA&E  as  a  tailored  AOA. 
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proposal  to  industry  to  submit  LCS  concepts.  The  Office  of  the  Secretary 
of  Defense  and  the  Joint  Staff  were  concerned  that  the  Navy’s  focus  on  a 
single  solution  did  not  adequately  consider  other  ways  to  address  littoral 
capability  gaps.  Based  on  these  concerns  in  late  2003,  the  Navy  was 
directed  to  consider  alternatives  to  surface  ships  such  as  submarines  and 
manned  aircrafts  in  the  ongoing  analysis  of  multiple  concepts.  The 
analysis,  which  was  led  by  the  Naval  Surface  Warfare  Center,  compared 
nonship  alternatives  to  LCS-concept  ships  and  concluded  that  the  LCS 
concept  remained  the  best  solution  to  provide  capabilities  in  the  littorals. 
However,  the  estimated  costs  for  the  various  LCS  ship  alternatives 
developed  in  the  analysis  far  exceeded  the  $220  million  (fiscal  year  2005 
dollars)  target  that  the  Navy  had  set  for  the  program.  The  Navy  stated  that 
because  the  cost  estimates  were  rough-order-of-magnitude  estimates  and 
were  based  on  preliminary  concept  designs,  those  costs  were  not  used  to 
make  cost  decisions  for  LCS.  However,  since  starting  development  in  2004, 
the  LCS  program  has  experienced  a  151  percent  growth  in  development 
costs  and  its  costs  are  closer  to  the  cost  estimates  from  the  analysis  of 
multiple  concepts  than  the  target  cost  set  by  the  Navy. 

DOD  and  service  officials  responsible  for  conducting  AOAs  indicated  that 
often  capability  requirements  are  proposed  that  are  so  specific  that  they 
effectively  eliminate  all  but  the  service  sponsor’s  preferred  concepts 
instead  of  considering  other  alternatives.  For  example,  in  recent  proposals 
to  address  a  global  strike  capability  need,  two  components  of  the  Air 
Force — the  Air  Combat  Command  and  Space  Command — defined  initial 
performance  requirements  that  required  two  different  approaches.  The  Air 
Force  Air  Combat  Command  defined  the  requirement  as  the  ability  to 
strike  a  target  within  1  day,  which  meant  that  bombers,  which  fall  under 
the  Air  Force  Air  Combat  Command’s  portfolio,  could  address  the  gap. 
However,  the  Air  Force  Space  Command  defined  the  requirement  in  the 
capability  proposal  as  the  ability  to  strike  a  target  within  a  certain  number 
of  hours,  which  meant  only  missiles,  which  fall  under  the  Air  Force  Space 
Command,  could  fulfill  the  need.  Although  OSD  PA&E  attempted  to  get 
the  Air  Force  to  consider  both  bombers  and  missiles  in  the  same  analysis, 
the  major  commands  argued  that  their  requirements  were  different  enough 
to  require  two  separate  analyses.  As  a  result,  the  Air  Force  Air  Combat 
Command  initiated  the  Next  Generation  Long-Range  Strike  AOA  for  a  new 
bomber,  while  the  Air  Force  Space  Command  initiated  the  Prompt  Global 
Strike  AOA  separately. 

Similarly,  for  the  ARH  AOA,  the  Army  called  for  very  specific  deployability 
requirements.  These  requirements  included  the  ability  to  fit  two 
helicopters  into  a  C-130  aircraft  and  for  the  helicopter  to  be  “fightable” 
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within  15  minutes  of  arrival.  The  Center  for  Naval  Analyses,  in  its  report 
on  the  factors  that  led  to  significant  cost  and  schedule  growth  in  the  ARH 
program,  noted  that  it  was  not  clear  whether  these  requirements  were 
needed  to  fulfill  the  operational  gap.  Furthermore,  the  Center  for  Naval 
Analyses  noted  that  due  to  the  stringent  deployability  requirements,  the 
program  had  effectively  eliminated  other  potentially  feasible  and  cost- 
effective  alternatives,  such  as  twin-engine  helicopters,  and  limited  the 
analysis  to  single  engine  alternatives. 


Timing  of  AO  As  May  Not  Many  AO  As  are  also  conducted  under  compressed  time  frames — 6  months 

Be  Conducive  to  Informing  or  less — or  concurrently  with  other  key  activities  that  are  required  for 
Trade  Offs  program  initiation,  in  order  to  meet  a  planned  milestone  decision  or 

weapon  system  fielding  date.  Consequently,  AOAs  may  not  have  enough 
time  to  assess  a  broad  range  of  alternatives  and  their  risks,  or  be 
completed  too  late  in  the  process  to  inform  effective  trade  discussions 
prior  to  beginning  development.  In  9  of  the  22  programs  we  reviewed  that 
had  AOAs,  the  timing  of  the  AOAs  was  compressed  or  concurrent  with 
other  planning  activities.  In  7  of  these  9  programs,  the  AOAs  were  limited. 
For  instance,  the  AOA  for  the  Future  Combat  Systems  program  was  a 
complex  undertaking;  however,  according  to  the  authors  of  the  AOA,  it 
was  conducted  in  half  the  time  that  a  less  complex  AOA  would  typically  be 
conducted.  In  addition,  due  to  schedule  constraints  imposed  to  meet  a 
preset  milestone  review  date,  the  AOA  was  performed  concurrently  with 
concept  development,  requirements  determination,  and  system  definition 
documents.  Ultimately,  the  Future  Combat  Systems  AOA  was  completed  1 
month  after  the  operational  requirements  were  validated  and  the  same 
month  that  the  program  was  approved  to  begin  system  development, 
which  precluded  trade  off  discussions  among  cost,  performance,  and  risks 
from  taking  place.  In  addition,  although  AOAs  are  required  to  be  done  for  a 
Milestone  B  decision,  the  Army’s  Warfighter  Information  Network-Tactical 
(WIN-T)  program  was  approved  to  begin  without  one.  The  milestone 
decision  authority  for  the  program  waived  the  AOA  requirement  until  a 
later  date.  The  WIN-T  AOA  was  completed  approximately  16  months  after 
the  program  started  (see  fig.  3). 
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Figure  3:  Milestones  for  WIN-T  Program 


Initial  Final  MS  B 

requirements  requirements  program  AOA 

approved  AOA  started  approved  initiation  completed 


Source:  GAO  presentation  of  DOD  data. 


DOD  Has  Not  Consistently  While  DOD  acquisition  policy17  requires  that  major  defense  acquisition 
Provided  Guidance  for  programs  conduct  an  AOA  prior  to  program  initiation  at  Milestone  B,  the 

Conducting  Individual  policy  does  not  specify  criteria  or  guidance  for  how  AOAs  should  be 

AO  As  conducted.  According  to  the  policy,  OSD  PA&E  is  to  provide  guidance  to 

programs  prior  to,  during,  and  after  their  AOA  has  been  completed.  The 
guidance  is  intended  to  ensure  that  the  services  are  examining  a  sufficient 
number  of  alternatives  that  take  into  consideration  joint  plans  and 
interoperability,  but  to  also  ensure  that  AOAs  are  analyzing  key  risks  such 
as  technology,  cost,  and  schedule.  In  9  of  the  22  programs  we  reviewed 
that  had  AOAs,  OSD  PA&E  either  provided  late  guidance  or  did  not 
provide  formal  guidance  when  AOAs  were  started.  In  6  of  these  9 
programs,  the  AOAs  were  limit  ed.  For  instance,  OSD  PA&E  did  not 
provide  guidance  for  the  AOA  that  supported  initiation  of  the  VH-71 
Presidential  Helicopter  program.  In  this  AOA,  the  service  had  very  specific 
performance  requirements  that  narrowed  the  scope  of  the  alternatives 
examined.  In  addition,  the  service  conducted  the  AOA  under  a  compressed 
schedule  to  meet  a  previously  planned  milestone,  which  may  not  have 
allowed  for  robust  analyses  of  technology  and  integration  risks.  These 
factors  most  likely  played  a  part  in  the  AOA  examining  only  3  alternatives 
and  eliminating  19  other  alternatives  early  on. 

DOD  officials  have  also  stated  that  when  OSD  PA&E  guidance  is  provided, 
it  is  sometimes  late.  For  example,  the  LCS  program  AOA  had  been 
underway  for  about  a  year  before  OSD  PA&E  provided  guidance  to  the 
Navy.  Officials  also  explained  that  guidance  is  often  informal,  sometimes 


17  DOD’s  acquisition  policy  is  contained  in  DOD  Instruction  5000.2,  the  Operation  of  the 
Defense  Acquisition  System. 
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provided  over  the  telephone,  or  if  written,  remains  in  draft  form  for  long 
periods,  preventing  the  services  from  formulating  and  having  analysis 
plans  approved.  However,  according  to  PA&E  officials,  sometimes 
guidance  is  never  formalized  or  written  because  the  services  do  not  have  a 
validated  capability  proposal  or  do  not  agree  with  the  scope  and  direction 
provided.  By  not  providing  timely  formal  guidance  before  AO  As  are 
started,  DOD  is  missing  an  opportunity  to  ensure  AOAs  examine  an 
appropriate  scope  of  alternatives  and  conduct  robust  risk  assessments. 


Recent  DOD  Policy 
Changes  Could 
Improve  AOA 
Effectiveness 


In  December  2008,  DOD  revised  its  acquisition  policy  and  introduced 
several  initiatives  based  in  part  on  direction  from  Congress  that  could 
provide  a  better  foundation  for  establishing  knowledge-based  business 
cases  for  initiating  weapon  system  programs.  The  revised  policy 
strengthens  the  front  end  of  the  acquisition  process  by  requiring  key 
systems  engineering  activities  and  early  prototyping,  and  establishing 
required  milestone  reviews  to  assess  whether  programs  are  acquiring  the 
requisite  knowledge  as  they  move  towards  the  start  of  system 
development.  In  addition,  in  March  2009,  DOD  revised  its  policy  governing 
the  JCIDS  process,  to  help  streamline  the  determination  of  capability 
needs  and  improve  the  integration  between  JCIDS  and  the  acquisition 
process.  In  revising  these  policies,  DOD  elevated  the  role  of  AOAs  in 
determining  weapon  system  concepts  and  strengthened  how  they  are  to  be 
implemented.  Improving  the  effectiveness  of  AOAs  will  depend  on  DOD’s 
ability  to  ensure  that  its  policy  changes  are  consistently  implemented  and 
reflected  in  decisions  on  individual  weapon  system  programs.  We  have 
reported  in  the  past  that  inconsistent  implementation  of  existing  policies 
has  hindered  DOD’s  efforts  to  plan  and  execute  programs  effectively.18  The 
key  revisions  to  the  policies  that  impact  AOAs  are  summarized  in  table  7. 


18  GAO,  Defense  Acquisitions:  DOD  Must  Balance  Its  Needs  with  Available  Resources  and 
Follow  an  Incremental  Approach  to  Acquiring  Weapon  Systems,  GAO-09-431T 
(Washington,  D.C.:  Mar.  3,  2009). 
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Table  7:  Policy  Revisions  That  Affect  AOAs 

DOD  5000.2  (May  2003  Revision) 

DOD  5000.02  (December  2008 

Revision) 

Differences 

AOA  required  by  regulation  for  Major 
Defense  Acquisition  Programs. 

AOA  required  by  statute  for  M  ilestone  B 
certification. 

Policy  now  reflects  congressional  legislation 
passed  in  2006  that  requires  the  milestone 
decision  authority  to  certify  to  Congress  that 
programs  approved  at  MS  B  have,  among 
other  things,  conducted  an  AOA. 

AOA  to  be  conducted  during  concept 
refinement  phase  leading  up  to  Milestone 

A  review  that  may  be  waived. 

AOA  to  be  conducted  during  Materiel 
Solution  phase  leading  up  to  Milestone  A 
review  that  must  be  certified  by  the 
milestone  decision  authority. 

Policy  now  places  greater  emphasis  on 
Milestone  A.  This  should  help  ensure  AOAs 
are  conducted  earlier. 

Focus  of  the  AOA  is  to  refine  the  selected 
concept  documented  in  the  approved 
capability  proposal  document. 

AOA  is  to  assess  potential  materiel 
solutions  to  satisfy  the  capability  need. 

AOA  intended  now  to  conduct  broader 
examination  of  alternatives. 

PA&E  to  provide  guidance  prior  to  the 
start  of  AOA. 

PA&E  to  provide  AOA  guidance  for 
approval  by  the  milestone  decision 
authority. 

AOA  guidance  now  required  to  be  approved 
by  milestone  decision  authority. 

AOA  study  plan  to  be  prepared  for  review 
by  PA&E. 

AOA  study  plan  to  be  approved  by  PA&E. 

AOA  study  plan  must  now  be  approved  by 
PA&E. 

AOA  shall  assess  the  critical  technologies 
associated  with  concepts,  including 
technology  maturity,  technical  risk,  and,  if 
necessary,  technology  maturation  and 
demonstration  needs. 

The  AOA  shall  assess  the  critical 
technology  elements  associated  with  each 
proposed  materiel  solution,  including 
technology  maturity,  integration  risk, 
manufacturing  feasibility,  and,  where 
necessary,  technology  maturation  and 
demonstration  needs. 

Policy  broadens  the  focus  of  AOA  risk 
assessments  to  include  integration  risk  and 
manufacturing  feasibility. 

Chairman  of  the  Joint  Chiefs  of  Staff 
Instruction  3170.01  F  (May  2007 
Revision) 

Chairman  of  the  Joint  Chiefs  of  Staff 
Instruction  31 70.01  G  (March  2009 
Revision) 

Differences 

If  a  materiel  solution  is  needed,  sponsor 
identifies  a  potential  solution  or  set  of 
solutions  to  fill  a  capability  gap. 

If  a  materiel  solution  is  needed,  sponsor 
will  make  recommendation  on  the  broad 
type  of  materiel  solution  preferred- 
information  system,  evolutionary 
development  of  an  existing  capability,  or  a 
transformational  approach. 

Policy  change  should  help  mitigate  sponsors 
identifying  a  preferred  solution  early  on  in  a 
proposed  capability  document. 

Source:  GAO  analysis  of  DOD  5000  and  CJ  CSI  3170  policies. 


DOD’s  revised  policies,  for  example,  may  help  mitigate  service  sponsors 
from  locking  into  a  solution  too  early  in  the  process  by  eliminating  the 
functional  solutions  analysis  in  a  capability  proposal,  which  identified  a 
preferred  solution  and  influenced  the  scope  of  alternatives  in  an  AOA.  In 
the  revision,  the  capability  proposal  will  only  identify  a  broad  category  of 
the  type  of  materiel  solution  that  should  be  considered;  for  example, 
whether  it  should  be  an  incremental  or  transformational  development 
approach.  The  AOA  will  then  assess  potential  solutions  as  determined  by 
the  milestone  decision  authority  and  within  the  broad  category 
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recommended.  This  change  integrates  essentially  what  had  been  two 
separate  trade  space  analyses  into  one  analysis.  In  doing  so,  it  sets  up  a 
better  opportunity  for  a  more  robust  analysis  of  alternatives. 

DOD’s  revised  acquisition  policy  also  now  imposes  early  milestone 
reviews  which  should  help  resolve  the  timing  issues  we  found  with  several 
AOAs  in  the  past.  Under  the  previous  policy,  AOAs  were  required  for 
program  initiation  at  Milestone  B,  which  may  have  led  to  some  AOAs  being 
completed  just  prior  or  even  after  program  initiation.  Under  the  revised 
policy,  AOAs  are  generally  required  earlier  in  the  process.  Furthermore, 
DOD  PA&E  is  required  to  be  involved  much  earlier  in  the  process  by 
providing  requisite  guidance  at  the  Materiel  Development  Decision  as  well 
as  approving  AOA  study  plans  before  an  AOA  is  started.  These  additional 
reviews  with  required  guidance  earlier  in  the  acquisition  process  should 
help  mitigate  conducting  AOAs  under  compressed  time  frames.  However, 
while  the  revised  policy  strengthens  the  front  end  of  the  acquisition 
process,  the  AOA  is  still  constrained  to  a  given  set  of  requirements  that 
may  be  unfeasible  and  could  lead  to  unsuccessful  program  outcomes,  such 
as  with  the  Armed  Reconnaissance  Helicopter  and  Future  Combat 
Systems. 


Conclusions 


With  increased  demand  and  competition  for  funding,  it  is  critical  that  DOD 
weapon  system  programs  provide  the  best  value  to  the  warfighter  and  to 
the  taxpayer.  Yet  in  too  many  cases,  DOD  programs  do  not  accomplish  this 
and  experience  significant  cost,  schedule,  and  performance  problems. 
Many  of  these  problems  could  be  avoided  if  programs  started  with  sound, 
knowledge-based  business  cases.  A  key  to  developing  such  business  cases 
is  having  effective  AOAs  that  analyze  and  compare  the  performance,  costs, 
and  risks  of  competing  solutions,  and  identify  the  most  promising  weapon 
system  solution  to  acquire.  The  majority  of  AOAs  we  reviewed  were 
limited  and  thus  did  not  sufficiently  inform  the  business  case  for  starting 
new  programs. 

DOD’s  recent  policy  revisions  are  positive  steps  that  could,  if  implemented 
properly,  provide  a  better  foundation  for  conducting  AOAs  and 
establishing  sound  business  cases  for  starting  acquisition  programs.  The 
revisions,  for  example,  should  help  ensure  that  DOD  direction  is  provided 
before  AOAs  are  started  and  that  AOAs  are  conducted  at  an  early  point  in 
the  acquisition  process  where  their  results  can  inform  key  decisions 
affecting  program  initiation.  However,  these  policy  changes  alone  will  not 
be  sufficient  to  ensure  AOAs  achieve  their  intended  objectives.  Unless 
mechanisms  are  established  to  ensure  policy  is  followed,  specific  guidance 
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and  criteria  are  developed  for  how  AOAs  should  be  conducted,  and  AOAs 
are  completed  before  program  requirements  are  set,  AOAs  will  not  provide 
effective  in-depth  analyses  and  DOD  will  continue  to  struggle  to  make 
informed  trade  offs  and  start  executable  programs. 


Recommendations 
Executive  Action 


j.  To  further  strengthen  the  effectiveness  of  AOAs  in  helping  DOD  establish 
sound  business  cases  for  major  weapon  programs,  we  recommend  that  the 
Secretary  of  Defense  take  the  two  following  actions: 


•  Establish  specific  criteria  and  guidance  for  how  AOAs  should  be 
conducted,  including  how  technical  and  other  programmatic  risks  should 
be  assessed  and  compared. 

•  Ensure  that  AOAs  are  completed  and  approved  before  program 
requirements — key  performance  parameters  and  attributes — are  finalized 
and  approved. 


Agency  Comments 
and  Our  Evaluation 


In  written  comments  on  a  draft  of  this  report,  DOD  concurred  with  our 
recommendations.  DOD’s  response  is  reprinted  in  appendix  II.  DOD  stated 
in  response  to  our  first  recommendation  that  it  had  made  significant 
progress  in  establishing  criteria  and  guidance  for  conducting  AOAs,  and  in 
defining  the  relationship/role  of  the  AOA  in  both  the  acquisition  and 
capabilities  determination  processes.  DOD  indicated  that  the  role  of  the 
AOA  has  been  defined  in  recently  revised  acquisition  policy  (Department 
of  Defense  Instruction  5000.02,  dated  Dec.  2,  2008)  and  capabilities  policy 
(Chairman,  Joint  Chiefs  of  Staff  Instruction  3170.01G,  dated  Mar.  1,  2009). 
While  we  agree  that  promising  improvements  have  been  made  in  revising 
the  policies,  they  do  not  go  far  enough  in  providing  specific  criteria  and 
guidance  for  how  AOAs  should  be  conducted.  Without  such  direction, 
there  is  a  risk  that  AOAs  will  continue  to  provide  limited  assessments  of 
weapon  system  options,  and  DOD  will  initiate  programs  without  sound, 
executable  business  cases.  In  concurring  with  our  second 
recommendation — that  AOAs  be  completed  before  requirements  are 
finalized — DOD  pointed  out  that  under  its  revised  acquisition  policy,  AOAs 
are  now  required  to  be  completed  before  the  formal  initiation  of  an 
acquisition  program.  We  agree  that  the  policy  should  help  improve  the 
timing  of  AOAs  so  that  they  are  conducted  at  an  early  point  in  the 
acquisition  process  and  provide  an  opportunity  for  trade  offs  to  take  place. 
However,  establishing  and  approving  requirements  is  another  key  step 
required  for  initiating  an  acquisition  program  and  this  is  done  under  a 
separate  process — the  Joint  Capabilities  Integration  and  Development 
System.  We  believe  that  DOD  needs  to  take  steps  to  ensure  that  program 
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requirements  are  not  finalized  before  the  AOA  is  completed  and  that  the 
results  of  the  AOA  are  used  to  inform  the  setting  of  requirements. 

DOD  also  provided  technical  comments,  which  we  incorporated  where 
appropriate. 


As  agreed  with  your  offices,  unless  you  publicly  announce  the  contents  of 
this  report  earlier,  we  plan  no  further  distribution  until  30  days  from  the 
report  date.  At  that  time,  we  will  send  copies  of  this  report  to  the  Secretary 
of  Defense;  the  Secretaries  of  the  Air  Force,  Army,  and  Navy;  and 
interested  congressional  committees.  This  report  will  also  be  available  at 
no  charge  on  the  GAO  Web  site  at  http://www.gao.gov. 

If  you  have  any  questions  about  this  report  or  need  additional  information, 
please  contact  me  at  (202)  512-4841  or  sullivanm@gao.gov.  Contact  points 
for  our  Offices  of  Congressional  Relations  and  Public  Affairs  may  be  found 
on  the  last  page  of  this  report.  GAO  staff  who  made  major  contributions  to 
this  report  are  listed  in  appendix  IV. 

Sincerely  yours, 


Michael  J.  Sullivan,  Director 
Acquisition  and  Sourcing  Management 
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Appendix  I:  Scope  and  Methodology 


To  assess  whether  analyses  of  alternatives  (AOA)  have  been  effective  in 
identifying  the  most  promising  options  and  providing  a  sound  rationale  for 
program  initiation,  we  analyzed  data  and  documents  for  Acquisition 
Category  (ACAT)  I  programs  that  have  been  initiated  between  fiscal  years 
(FY)  2003  and  2008  and  were  in  the  Department  of  Defense’s  (DOD)  FY 
2008  Major  Defense  Acquisition  Program  (MDAP)  list.  The  relevant  policy 
that  governs  the  AOA  process  for  these  programs,  DOD  Instruction  5000.2 
( Operation  of  the  Defense  Acquisition  System),  was  revised  by  DOD  in 
May  of  2003  and  revised  again  in  December  of  2008  to  become  DOD 
Instruction  5000.02.  As  a  result,  we  used  the  May  2003  DOD  Instruction  to 
assess  the  AOAs.  Using  DOD’s  FY  2008  MDAP  list  and  Milestone  B  dates 
provided  by  DOD,  we  identified  34  ACAT  I  programs  that  had  been 
initiated,  or  started  system  development  and  production,  between  2003 
and  2008.  Programs  that  had  been  initiated  between  2003  and  2008  but 
were  not  in  the  FY  2008  MDAP  list,  such  as  programs  terminated  before 
2008,  were  not  included  in  the  analysis.  We  collected  AOA  full  reports, 
executive  summaries,  guidance  documents,  and  study  plans  when 
available,  from  program  officials.  Program  officials  also  responded  to  data 
collection  surveys  we  distributed  through  service  action  officers  to  gather 
information  about  their  programs’  AOA,  guidance,  capability  documents, 
and  how  the  AOA  led  to  changes  to  the  program  concept.  An  official  for 
the  Cobra  Judy  Replacement  program  responded  to  the  survey,  but 
officials  did  not  respond  to  several  phone  calls  and  e-mails  requesting 
additional  documentation,  so  this  program  was  not  included  in  the 
analysis.  In  addition,  because  the  Combat  Search  and  Rescue  Replacement 
Vehicle  (CSAR-X)  program  did  not  start  development. 

Of  the  remaining  32  programs,  10  programs  did  not  have  AOAs.  Whether  a 
program  had  an  AOA  or  not  was  determined  through  analysis  of  program 
documents  and  survey  responses.  For  the  22  programs  that  had  AOAs, 
program  documents  and  survey  data  were  reviewed  to  determine  the 
scope  of  the  AOAs  and  whether  the  AOA  assessed  technology  and 
integration  risks.  An  AOA’s  scope  was  assessed  to  be  narrow  if  the  AOA 
examined  2  to  5  alternatives  within  a  single  concept  and  assessed  to  be 
broad  if  the  AOA  examined  8  to  26  alternatives  within  a  single  concept  or 
multiple  concepts.  An  AOA  was  assessed  to  have  not  completed  any  risk 
analyses  for  its  alternatives  when  it  made  no  mention  of  risks  in  the  entire 
AOA  report;  assessed  to  be  limited  if  the  risk  analyses  were  not  completed 
for  all  of  the  alternatives,  if  integration  risks  were  not  examined,  or  if  the 
risk  analyses  were  not  emphasized  in  the  conclusions  and 
recommendations;  and  assessed  to  be  adequate  if  technical  and  integration 
risks  were  analyzed  and  compared  for  all  of  the  alternatives.  We  followed 
up  with  some  program  officials  through  phone  calls  and  e-mails  for 
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additional  information.  To  assess  how  the  quality  of  AO  As  correlates  with 
programs’  outcomes,  we  also  collected  program  and  cost  data  from  DOD’s 
Selected  Acquisition  Reports  and  GAO’s  Annual  Assessments  of  Selected 
Weapon  Programs.  Programs  with  less  than  10  percent  cost  growth  were 
considered  to  have  low  cost  growth,  programs  with  10  to  24  percent  cost 
growth  were  considered  to  have  moderate  cost  growth,  and  programs  with 
25  percent  or  more  cost  growth  were  considered  to  have  high  cost  growth. 
Programs  with  less  than  7  months  of  delay  in  initial  operational  capability 
or  acquisition  cycles  were  considered  to  have  low  schedule  growth, 
programs  with  7  tol2  months  of  delay  in  initial  operational  capability  or 
acquisition  cycles  were  considered  to  have  moderate  schedule  growth, 
and  programs  with  greater  than  12  months  of  delay  in  initial  operational 
capability  or  acquisition  cycles  were  considered  to  have  high  schedule 
growth.  The  32  programs  we  reviewed  accounted  for  one  third  of  the  96 
programs  in  DOD’s  2008  Major  Defense  Acquisition  Program  portfolio  and 
approximately  22  percent  of  the  total  planned  funding  commitments. 

To  identify  the  factors  that  have  affected  the  scope  and  quality  of  AO  As, 
we  reviewed  program  documents,  analyzed  data  from  the  survey,  and 
reviewed  DOD  policy.  We  reviewed  Initial  Capabilities  Documents  (ICD) 
gathered  from  the  Joint  Staffs  Knowledge  Management/Decision  Support 
tool  and  AO  As  to  determine  how  preferred  solutions  were  carried  from  the 
requirements-generation  process  to  the  acquisition  process.  To  determine 
how  program  schedules  affected  AOA  scope  and  methodology,  we 
analyzed  AOA  documents,  program  milestone  dates,  and  AOA  completion 
dates.  To  assess  how  DOD  study  guidance  affected  the  quality  of  AO  As,  we 
analyzed  whether  DOD  provided  guidance  through  survey  responses  and 
followed  up  with  DOD  to  confirm  those  responses.  We  also  reviewed 
regulations  and  policies  issued  by  the  Joint  Staff,  the  military  services,  and 
DOD,  as  well  as  other  DOD-produced  documentation  related  to  AOAs. 

To  determine  what  additional  actions  may  be  needed  to  address  the 
limitations  in  the  AOA  process,  we  analyzed  relevant  DOD  policies  and 
federal  statutes,  including  DOD  Instruction  5000.2  (May  2003),  DOD 
Instruction  5000.02  (December  2008),  the  Chairman  of  the  Joint  Chiefs  of 
Staff  Manual  (CJCSM)  3170.01C  (May  2007),  CJCSM  3170.01  (March  2009), 
and  Section  2366a  of  Title  10,  United  States  Code. 

In  researching  all  three  objectives,  we  interviewed  officials  from  the  U.S. 
Army  G3;  U.S.  Army  Training  and  Doctrine  Command  Analysis  Center 
(TRAC);  U.S.  Army  Capabilities  Integration  Center  (ARCIC);  U.S.  Air  Force 
Office  of  Aerospace  Studies;  Office  of  the  Assistant  Secretary  for 
Acquisition,  Deputy  Assistant  Secretary  of  the  Air  Force  for  Science, 
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Technology,  and  Engineering;  Air  Force  Acquisitions  -  Global  Reach; 
Deputy  Assistant  Secretary  of  the  Navy,  Acquisition  and  Logistics 
Management  (A&LM);  Deputy  Directorate  for  Antiterrorism  and  Homeland 
Defense,  J-34,  Joint  Staff;  Office  of  the  Secretary  of  Defense,  Acquisition, 
Technology  &  Logistics;  Office  of  the  Secretary  of  Defense,  Program 
Analysis  and  Evaluation;  Office  of  the  Deputy  Under  Secretary  of  Defense 
for  Science  and  Technology  (Acquisition  and  Technology)/Systems  and 
Software  Engineering;  Armed  Reconnaissance  Helicopter  Product 
Manager’s  Office;  U.S.  Army  Aviation  Center;  Deputy  Assistant  Secretary 
of  the  Navy,  Ship  Programs;  Littoral  Combat  Ship  Program  Office;  Marine 
Corps  Combat  Development  Command;  Office  of  the  Chief  of  Naval 
Operations,  Deputy  Chief  of  Naval  Operations,  Integration  of  Capabilities 
and  Resources  (N8),  Director  of  Warfare  Integration  (N8F),  Director  of 
Surface  Warfare  (N86);  Air  Combat  Command/A8I  (Requirements), 
Secretary  of  the  Air  Force  Technical  and  Analytical  Support;  and  the  Naval 
Surface  Warfare  Center,  Dahlgren. 

We  conducted  this  performance  audit  from  June  2008  to  September  2009 
in  accordance  with  generally  accepted  government  auditing  standards. 
Those  standards  require  that  we  plan  and  perform  the  audit  to  obtain 
sufficient,  appropriate  evidence  to  provide  a  reasonable  basis  for  our 
findings  and  conclusions  based  on  our  audit  objectives.  We  believe  that 
the  evidence  obtained  provides  a  reasonable  basis  for  our  findings  and 
conclusions  based  on  our  audit  objectives. 
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OFFICE  OF  THE  UNDER  SECRETARY  OF  DEFENSE 

3000  DEFENSE  PENTAGON 
WASHINGTON,  DC  20301-3000 


ACQUISITION 

TECHNOLOGY 

AND  LOGISTICS  $EP  —  3  2009 

Mr.  Michael  J.  Sullivan 

Director,  Acquisition  and  Sourcing  Management 
U.S.  Government  Accountability  Office 
441  G  Street,  N.W. 

Washington,  DC  20548 

Dear  Mr.  Sullivan: 

This  is  the  Department  of  Defense  (DoD)  response  to  the  GAO  draft  report 
GAO-09-665,  “DEFENSE  ACQUISITIONS:  Many  Analyses  of  Alternatives  Have  Not 
Provided  a  Robust  Assessment  of  Weapon  System  Options,”  dated  August  5,  2009 
(GAO  Code  120747). 

The  DoD  concurs  with  both  recommendations.  The  rationale  for  the  DoD’s 
position  is  enclosed. 

We  appreciate  the  opportunity  to  comment  on  the  draft  report.  My  point  of 
contact  for  this  effort  is  Mr.  Ron  Woods,  703-697-8183,  ronald.woods@osd.mil. 

Sincerely, 

David  G.  Ahern 
Director 

Portfolio  Systems  Acquisition 

Enclosure: 

As  stated 
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GAO  Draft  Report  Dated  August  5,  2009 
GAO-09-665  (GAO  CODE  120747) 


“DEFENSE  ACQUISITIONS:  Many  Analyses  of  Alternatives  Have  Not 
Provided  a  Robust  Assessment  of  Weapon  System  Options” 

DEPARTMENT  OF  DEFENSE  COMMENTS 
TO  THE  GAO  RECOMMENDATIONS 


RECOMMENDATION  1:  The  GAO  recommends  that  the  Secretary  of  Defense 
establish  specific  criteria  and  guidance  for  how  analyses  of  alternatives  should  be 
conducted,  including  how  technical  and  other  programmatic  risks  should  be  assessed  and 
compared. 

POD  RESPONSE:  Concur.  DoD  has  already  made  significant  progress  in  establishing 
the  criteria  and  guidance  for  the  conduct  of  an  Analysis  of  Alternatives  (AoA),  and  in 
defining  the  relationship/role  of  the  AoA  in  both  the  acquisition  and  capabilities 
validation  processes.  The  emergent  role  of  the  AoA  has  been  defined  as  providing  the 
key  analytical  foundation  for  assessing  the  utility  of  options  available  to  the  Department 
in  addressing  validated  capability  requirements  by  conducting  an  analysis  that  trades  off 
cost,  schedule,  performance  and  risk  (both  mission  and  programmatic  -  to  include 
technical  risk)  among  the  various  alternative  options.  This  role  has  been  codified  in  the 
recently  signed  Department  of  Defense  Instruction  5000.02,  “Operation  of  the  Defense 
Acquisition  System,”  dated  December  2,  2008,  and  in  the  Chairman,  Joint  Chiefs  of  Staff 
Instruction,  “Joint  Capabilities  Integration  and  Development  System,”  dated  March  1, 
2009.  The  Department  is  continuing  to  revise  policy  and  procedures  regarding  the  AoA 
in  response  to  the  recently  enacted  Weapon  System  Acquisition  Reform  Act  of  2009. 

RECOMMENDATION  2:  The  GAO  recommends  that  the  Secretary  of  Defense  ensure 
that  analyses  of  alternatives  are  completed  and  approved  before  program  requirements- 
key  performance  parameters  and  attributes-are  finalized  and  approved. 

DOD  RESPONSE:  Concur.  Guidance  for  the  AoA,  with  the  Director/Cost  Assessment 
and  Program  Evaluation  (D,CAPE)  leading  the  development  of  the  guidance,  has  become 
an  accepted  aspect  of  DoD  guidance.  The  Material  Development  Decision  (MDD)  is 
now  the  mandatory  entry  point  into  the  acquisition  system  with  an  approved  Initial 
Capabilities  Document  and  D,CAPE  drafted  AoA  guidance  as  necessary  entry  conditions. 
The  MDD  occurs  before  the  formal  initiation  of  an  acquisition  program.  The  completed 
AoA  is  now  required  before  the  initiation  of  an  acquisition  program  and  should  provide 
an  analytically  relevant  process  for  assessing  military  utility,  affordability  and  risk. 
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We  surveyed  32  major  defense  acquisition  programs  on  their  analyses  of 
alternatives  process  and  outputs.  Ten  of  the  programs  did  not  conduct 
AO  As.  The  following  table  provides  characteristics  of  the  22  programs  that 
conducted  AOAs.1 


Table  8:  Key  Characteristics  of  Programs  That  Conducted  AOAs 

Program 

Acquisition 

category 

Service 

sponsor 

Organization 

conducting 

AOA 

Type  of 
organization 

Length  of  AOA 
(in  months) 

Joint 

Months 
between  AOA 
completion 
and  program 
start 

AGM-88E  AARGM  - 
AG M-88E  Advanced 
Anti-Radiation  G uided 
Missile  (AARGM) 
Program 

1C 

Navy 

Center  for  Naval 
Analyses  (CNA); 
Naval  Air 

Warfare  Center, 
China  Lake 

FFRDC  and 
Service 

Cannot  be 
Determined 

No 

Cannot  be 
Determined 

AB3  -  Apache  Block  III 

ID 

Army 

Army's  TRADOC 
Analysis  Center 
(TRAC) 

Service 

6 

No 

18 

ARH  -  Armed 

Reconnaissance 

Helicopter 

ID 

Army 

Army's  TRADOC 
Analysis  Center 
(TRAC) 

Service 

6 

No 

6 

BAMS  -  Broad  Area 
Maritime  Surveillance 

ID 

Navy 

MITRE 

Corporation 

FFRDC 

15 

No 

55 

CVN  21  -  Next 
Generation  Nuclear 
Aircraft  Carrier 

ID 

OSD  with 

Navy 

Support 

Center  for  Naval 
Analyses  (CNA) 

FFRDC 

46 

Yes 

52 

DDG  1000  - 
ZUMWALT  CLASS 
Destroyer 

ID 

Navy 

Center  for  Naval 
Analyses  (CNA) 

FFRDC 

39 

No 

92 

E-2D  AHE  -  E-2D 
Advanced  Hawkeye 

ID 

Navy 

Naval  Research 
Lab  and  Whitney, 
Bradley  &  Brown, 
Inc. 

Service  and 

Contractor 

32 

No 

7 

1  We  removed  the  CSAR-X  program  from  our  analysis  because  the  program  was  terminated 
before  contracts  were  awarded. 
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Program 

Acquisition 

category 

Service 

sponsor 

Organization 

conducting 

AOA 

Type  of 
organization 

Length  of  AOA 
(in  months) 

Joint 

Months 
between  AOA 
completion 
and  program 
start 

EA-18G  -  Electronic 
attack  variant  of  the 
F/A-18  aircraft 

ID 

OSD  with 

Navy 

Support 

Various, 

including  military 
and  government 
agencies  as  well 
as  the  J  ohns 
Hopkins 

University 

Applied  Physics 
Lab 

Multiple 

Services,  FFRDC 
and  Contractor 

29 

Yes 

24 

FCS  -  Future  Combat 
Systems 

ID 

Army 

Army's  TRADOC 
Analysis  Center 
(TRAC) 

Service 

15 

No 

0 

CH-53K  -  Heavy  Lift 
Replacement  Program 

ID 

Navy 

Booz  Allen 
Hamilton 

Contractor 

5 

No 

27 

J  CA  -  J  ointCargo 
Aircraft 

ID 

Army 

Army's  TRADOC 
Analysis  Center 
(TRAC) 

Service 

9 

No 

24 

J  LENS  -  J  oint  Land 
Attack  Cruise  Missile 
Defense  Elevated 
Netted  Sensor  System 

ID 

Army 

Army's  TRADOC 
Analysis  Center 
(TRAC) 

Service 

17 

No 

34 

JTRS  AMF  -  J  oint 
Tactical  Radio  System 
Airborne  & 

Maritime/Fixed  Station 

ID 

ASD  Nil 
and  the 

J  oint  Staff 

Institute  for 
Defense 

Analyses  (IDA) 
and  the  J  ohns 
Hopkins  Applied 
Physics  Lab 

FFRDC  and 

University 

Affiliated 

Research  Center 

13 

Yes 

26 

LCS  -  Littoral  Combat 
Ship 

ID 

Navy 

Naval  Surface 
Warfare  Center, 
Dahlgren 

Service 

10 

No 

1 

LHA  REPLACEMENT 
-  New  Amphibious 
AssaultShip 

ID 

Navy 

Center  for  Naval 
Analyses  (CNA) 

FFRDC 

15 

No 

40 

LUH  -  Light  Utility 
Helicopter 

1C 

Army 

Army's  TRADOC 
Analysis  Center 
(TRAC) 

Service 

4 

No 

18 

MPS  -  Mission 

Planning  System 

ID 

Air  Force 

Air  Force  Air 

Combat 

Command 

Service 

14 

No 

54 

MUOS  -  Mobile  User 
Objective  System 

ID 

Navy 

Aerospace 

Corporation 

FFRDC 

28 

No 

30 

P-8A  -  Poseidon 
Program 

ID 

Navy 

Center  for  Naval 
Analyses  (CNA) 

FFRDC 

19 

No 

28 
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Program 

Acquisition 

category 

Service 

sponsor 

Organization 

conducting 

AOA 

Type  of 
organization 

Length  of  AOA 
(in  months) 

Joint 

Months 
between  AOA 
completion 
and  program 
start 

SDB  1  -  Small 

Diameter  Bomb 
Increment  1 

1C 

Air  Force 

Air  Combat 
Command  with 
support  from 
Boeing, 

Raytheon, 
Lockheed  Martin, 
and  Northrup 
Grumman 

Service 

contractors 

28 

Yes 

35 

VH-71  -  Presidential 
Helicopter  Fleet 
Replacement  Program 

ID 

Navy 

Center  for  Naval 
Analyses  (CNA) 

FFRDC 

17 

No 

15 

WIN-T  -  Warfighter 
Information  Network- 
Tactical 

ID 

Army 

Army's  TRADOC 
Analysis  Center 
(TRAC) 

Service 

25 

No 

16  months  after 
MS  B 

Source:  GAO. 
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GAO  Contact 


Michael  J.  Sullivan,  (202)  512-4841  or  sullivanm@gao.gov 


Staff 
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GAO’s  Mission 

The  Government  Accountability  Office,  the  audit,  evaluation,  and 
investigative  arm  of  Congress,  exists  to  support  Congress  in  meeting  its 
constitutional  responsibilities  and  to  help  improve  the  performance  and 
accountability  of  the  federal  government  for  the  American  people.  GAO 
examines  the  use  of  public  funds;  evaluates  federal  programs  and  policies; 
and  provides  analyses,  recommendations,  and  other  assistance  to  help 
Congress  make  informed  oversight,  policy,  and  funding  decisions.  GAO’s 
commitment  to  good  government  is  reflected  in  its  core  values  of 
accountability,  integrity,  and  reliability. 

Obtaining  Copies  of 
GAO  Reports  and 
Testimony 

The  fastest  and  easiest  way  to  obtain  copies  of  GAO  documents  at  no  cost 
is  through  GAO’s  Web  site  (www.gao.gov).  Each  weekday  afternoon,  GAO 
posts  on  its  Web  site  newly  released  reports,  testimony,  and 
correspondence.  To  have  GAO  e-mail  you  a  list  of  newly  posted  products, 
go  to  www.gao.gov  and  select  “E-mail  Updates.” 
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http://www.gao.gov/ordering.htm. 

Place  orders  by  calling  (202)  512-6000,  toll  free  (866)  801-7077,  or 

TDD  (202)  512-2537. 

Orders  may  be  paid  for  using  American  Express,  Discover  Card, 

MasterCard,  Visa,  check,  or  money  order.  Call  for  additional  information. 
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Congressional 

Relations 

Ralph  Dawn,  Managing  Director,  dawnr@gao.gov,  (202)  512-4400 
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